Integrated detection and tracking of multiple faces using particle filtering and optical flow-based elastic matching

The design and implementation of a multiple face tracking framework that integrates face detection and face tracking is presented. Specifically, the incorporation of a novel proposal distribution and object shape model within the face tracking framework is proposed. A general solution that incorporates the most recent observation in the proposal distribution using a multiscale elastic matching-based optical flow algorithm is proposed. The proposed multiscale elastic matching-based optical flow algorithm is shown to be general and powerful in three significant ways. First, it allows for the tracking of both, rigid and elastic objects. Second, it enables robust tracking even in the face of sudden and gradual changes in illumination, scale and viewpoint. Third, it is suitable for tracking using both, fixed cameras and moving cameras. The proposed object shape model is based on a kernel-based line segment matching algorithm, which incorporates a voting scheme similar to the Radon Transform. The incorporation of the object shape model is shown to improve the computational complexity and accuracy of the face tracking algorithm and also enhance its robustness to occlusion, noise and scene clutter. Efficient techniques for particle sampling based on the Genetic Algorithm and for computation of the region-based likelihood function using the integral image are proposed. The incorporation of face detection within the face tracking algorithm is also proposed. Experimental results show that the proposed face tracking system is very robust in its ability to handle occlusion, noise, scene clutter and changes in illumination, scale and viewpoint and is also computationally efficient. ! 2008 Elsevier Inc. All rights reserved.